Touchless fingerprinting acquisition and processing application for mobile devices

ABSTRACT

This disclosure is of a biometric authentication system and method. The system includes a mobile device having a camera and a screen, and a database. The system is programmed to superimpose on the screen an overlay of a finger over a real-time image seen by the camera, capture an image of a fingerprint of a user with the camera, compare the captured image with an authenticated fingerprint image that is stored in the database, and return a positive result if the compared images match.

I. BACKGROUND OF THE INVENTION

A. Field of Invention

This invention pertains to the art of biometric authentication and morespecifically to a biometric system that uses a mobile device with acamera to capture a fingerprint image.

B. Description of the Related Art

U.S. Pat. Pub. No. 2012/0086794 discloses a contactless fingerprintacquisition and processing method that includes the steps of detectingand acquiring an object image, converting the object image into afingerprint image and at least one of identifying and verifying thefingerprint image.

E.P.O. Pat. Pub. No. EP 2388734 discloses a photography subsystemconfigured to be operated in fingerprint scanning mode and a normalmode. An autofocus module may be initialized to control a lens with aninitial focal length set according to received instructions indicatingone of the modes of operation. An image signal processor associated withthe photography subsystem may process images received from thephotography subsystem and, if necessary, process the images to form acandidate digital fingerprint for use in an authentication process.

U.S. Pat. No. 5,828,773 discloses a fingerprint sensing device thatincludes a fingerprint sensor, a processor for determining an actualfingerprint position on the fingerprint sensor relative to a desiredfingerprint position, and a finger position indicator for generating afinger position indication to assist the user in positioning the fingerto the desired fingerprint position based upon the actual fingerprintposition on the fingerprint sensor. The processor calculates afingerprint center point defining the actual fingerprint position. Thefingerprint sensor may be an electric field sensor in integrated circuitform. The finger position indicator may be a visual indicator, such as aposition image indicia generator for generating image indicia on adisplay screen relating to the fingerprint position on the fingerprintsensor, and wherein an actual fingerprint position image is alsogenerated relative to the position image indicia on the display screen.The position image indicia may be a desired fingerprint center pointindicia image. Method steps of the design are also disclosed.

U.S. Pat. No. 8,312,291 discloses a biometric user authentication methodthat includes the steps of: enrolling a user based on user's biometricsamples to generate user's reference data; and authenticating the userbased on a user's live biometric sample and the user's reference data;wherein enrolling a user includes acquiring the user's biometricsamples; extracting an enrollment feature vector from each user'sbiometric sample; computing a biometric reference template vector as amean vector based on the enrollment feature vectors; computing avariation vector based on the enrollment feature vectors and the meanvector; randomly generating an enrollment secret vector; computing anenrollment code vector based on the enrollment secret vector and thevariation vector; computing a difference vector as a wrap-arounddifference between the enrollment code vector and the mean vector;computing an error correction vector based on the enrollment secretvector to enable error correction during the user authentication phaseaccording to a given error tolerance level, wherein the error correctionvector is not computed if the error tolerance level is equal to zero;and storing the variation vector, the difference vector, and the errorcorrection vector as a part of the user's reference data to be usedduring the user authentication phase.

U.S. Pat. Pub. No. 2008/0101664 discloses a method of recognizing 3Dfingerprints by contact-less optical means. The method comprises interalia the steps of: obtaining an optical contact-less means for capturingfingerprints, such that 3D optical images, selected from a groupcomprising minutia, forks, endings or any combination thereof areprovided; obtaining a plurality of fingerprints wherein the imagesresolution of said fingerprints is not dependent on the distance betweena camera and said inspected finger; correcting the obtained images bymis-focal and blurring restoring; obtaining a plurality of images,preferably 6 to 9 images, in the enrolment phase, under various viewsand angles; systematically improving the quality of the field depth ofsaid images and the intensity per pixel; and, disengaging higherresolution from memory consumption, such that no additional opticalsensor is required.

II. Summary

In accordance with one aspect of the present invention, a method forbiometric authentication of a user includes the steps of: a) providing amobile device including a camera and a screen, and a database; b)superimposing on the screen an overlay of a finger over a real-timeimage as currently seen through the camera; c) capturing with the cameraan image of a fingerprint of the user; d) comparing the captured imagewith an authenticated fingerprint image that is stored in the database;and e) returning a positive result if the compared images match.

In accordance with another aspect of the present invention, a method forbiometric authentication of a user includes the steps of: a) providing:a mobile device including a camera and a screen; a database; and arandomly-generated image that is unique to each user using the mobiledevice; b) capturing with the camera an image of a fingerprint of theuser; c) extracting first significant geographical areas of thefingerprint image; d) extracting second significant geographical areasof the randomly-generated image; e) concatenating the first significantgeographical areas with the second significant geographical areas toproduce a concatenated string; f) comparing the concatenated string withan authenticated concatenated string that is stored in the database; andg) returning a positive result if the compared strings match.

In accordance with another aspect of the present invention, a biometricauthentication system includes: a mobile device including a CCD or CMOScamera and a screen; and a database; wherein the system is programmedto: superimpose on the screen an overlay of a finger over a real-timeimage seen by the camera; capture an image of a fingerprint of a userwith the camera; compare the captured image with an authenticatedfingerprint image that is stored in the database; and return a positiveresult if the compared images match.

Still other benefits and advantages of the invention will becomeapparent to those skilled in the art to which it pertains upon a readingand understanding of the following detailed specification.

III. BRIEF DESCRIPTION OF THE DRAWINGS

The invention may take physical form in certain parts and arrangement ofparts, embodiments of which will be described in detail in thisspecification and illustrated in the accompanying drawings which form apart hereof and wherein:

FIG. 1 is a diagram of a biometric system.

FIG. 2 is a flow chart of a biometric system method.

FIG. 3 is a front view of a mobile device capturing a user's fingerprintimage.

FIG. 4 is a flow chart of a method of creating an overlay guide from auser's finger image.

FIG. 5 is a flow chart of a method of concatenating a user's fingerprintimage with a random image file in a biometric system.

IV. DETAILED DESCRIPTION OF THE INVENTION

Referring now to the drawings wherein the showings are for purposes ofillustrating embodiments of the invention only and not for purposes oflimiting the same, and wherein like reference numerals are understood torefer to like components, FIG. 1 shows a diagram of a biometric system10 according to an embodiment of this invention. This embodiment mayinclude a user 12, a mobile device 14, and a server 18. The mobiledevice 14 may include a camera 16. The server 18 may include a database20. In this embodiment, the user 12 may use the camera 16 of the mobiledevice 14 to capture an image 30 of the user's finger 22 that definesthe fingerprint. The finger 22 that defines the fingerprint may but doesnot need to contact the mobile device 14 or the camera 16 to capture theimage 30. The image 30 of the fingerprint may be transmitted to theserver 18 where the image 30 may be compared to other images in thedatabase 20 to determine whether the image 30 matches a database entry.The server 18 may communicate the results back to the mobile device 14.The comparison may be implemented by, for example, performing standarddeviation analysis or other known techniques.

The mobile device 14 may include, but is not limited to, a cell phone,an MP3 player, a tablet PC, a laptop computer, or a personal digitalassistant. In alternative embodiments of the invention, the softwareimplementing the system 10 on the mobile device 14 may operate onAndroid, iOS, Windows 7, or Windows 8 operating systems. One of ordinaryskill in the art would understand that the biometric system 10 may alsobe used with non-mobile devices that include a camera, for example acomputer with a web-camera. One of ordinary skill in the art would alsounderstand that the biometric system 10 may work with other biometricindicators of a user 12 other than fingerprints, including, but notlimited to: face recognition, iris smays, palm prints, voice signatures,and/or signature comparison. The biometric system 10 may be used for,but is not limited to, unlocking the mobile device 14, accessingapplications on the mobile device 14, or authentication of mobilepayment systems.

One of ordinary skill in the art would understand that the server 18 maybe a general purpose computer programmed with specific software toimplement the method of this biometric system 10. Alternativeembodiments of the system may use a special-purpose computer, amini-computer, a mainframe computer, or a workstation. The server 18 maybe a cloud-based network of computers. In another embodiment, the server18 may be an HTTP web server. Communication between the mobile device 14and server 18 may be by any method known to one of ordinary skill in theart, including, but not limited to, wired communication, LAN, Ethernet,USB, Firewire, wireless communication, WiFi, Bluetooth, RF, CDMA, TDMA,GSM, and/or infrared. In another embodiment of the invention, thedatabase 20 may be stored on the mobile device 14, so that no externalserver 18 is required. In such an embodiment, all of the steps of FIG. 2may be performed by the mobile device 14. In another embodiment, thedatabase 20 may be a SQLite database.

FIG. 2 shows a flow chart of one embodiment of a biometric systemmethod. At block/step 32, the biometric system 10 may be activated. Suchactivation may occur on initiation by the user 12. Alternatively, suchactivation may occur when the mobile device 14 is powered on.Alternatively, such activation may be triggered by other software suchas, but not limited to, a request for authentication by a mobile paymentsystem.

At block/step 34, the system 10 may initialize. During suchinitialization, the mobile device 14 may turn on the camera 16 andprepare to capture an image. At this block/step 34, the system 10 mayask the user 12 to input the user's name or other identification.Alternatively, the system 10 may allow the user 12 to indicate that theuser 12 is a new user and that enrollment of the user 12 is necessary.In such a case, the user 12 may input the user's name or otheridentification into the system 10. Alternatively, the system 10 mayproceed without an existing enrolled user 12 inputting the user's nameor other identification. Alternatively, the device 14 may remember thename or other identification of the last user 12 to have used the device14 and assume that the same user 12 is now using the system 10.

At block/step 36, the user 12 may position the user's finger 22 beforethe camera 16. In another embodiment, the device 14 may automaticallyset the focal depth, exposure, and light level depending on thepositioning of the user's finger 22 before the camera 16 and ambientconditions.

At block/step 38, the camera 16 may capture an image 30 of the user'sfinger 22. Such capture may be triggered automatically when the mobiledevice software indicates that a finger 22 is positioned in front of thecamera 16. Alternatively, the capture may be triggered by the user 12pressing a button on the device 14. It is understood to one of ordinaryskill in the art that such a button may be a virtual or soft button. Inanother embodiment of the invention, the mobile device 14 may mask orblack out the top third of the field of view of the camera 16 and/or thebottom third of the field of view of the camera 16 during block/step 36.If the user 12 positions the finger 22 horizontally in front of thecamera 16, such masking will not prevent capturing the fingerprint image30 but may reduce any background captured, may decrease the size of thecaptured image 30, and may increase transmission or processing times forthe image 30.

At block/step 40, the captured image 30 may be processed. In variousembodiments, such processing may include, but is not limited to, one ormore of the following: converting the image 30 to a black and whiteimage; cropping the top third of the image 30 and/or the bottom third ofthe image 30; reducing image noise; reducing any blurring of the image30; enhancing ridge contours of the image 30; compressing the image 30;encrypting the image 30; and binarizing the fingerprint image 30. Inanother embodiment, at block/step 40, the captured image 30 may beanalyzed to determine whether it is of sufficient quality for use in thesystem 10. If not, the user 12 may be prompted to again capture an image30 of the user's fingerprint. In another embodiment, if processing iscompleted in a faster time than average, the system 10 may return anegative result; the underlying premise may be that an unauthorizedperson may have placed a two-dimensional photograph of a fingerprintbefore the camera 16, which may require less processing than for athree-dimensional real finger 22.

At block/step 42, the processed image may be transmitted for enrollment,verification, or identification. Such transmission may be to the server18 if the database 20 is not on the mobile device 14, or suchtransmission may be internally within the device 14 to its database 20if the database is stored on the device 14. In another embodiment of theinvention, the transmission step 42 and the processing step 40 may beswitched so that transmission occurs first, and then processing occurson the server 18.

At block/step 44, the system 10 may determine which mode to operate in:enrollment of a new user, verification of an identified user, oridentification of a user from the enrolled user's database. The user 12may select what mode the system 10 is to operate in, or the mobiledevice 14 may be pre-set to operate in a certain mode. Alternatively,different external triggers may set the mode of the system 10 dependingon specific pre-programmed conditions.

At block/step 46, the system 10 may operate in the enrollment mode. Atthis block/step 46, the system may save the fingerprint image 30 to thedatabase 20. In another embodiment, the system 10 may prompt the user 12to enter other identifying information into the mobile device 14 at thisstep. In another embodiment, the mobile device 14 may capture multipleimages 30 of the user's finger 22 during enrollment and save themultiple images 30 to the database 20. These multiple images 30 may betaken under varying ambient conditions. These multiple images 30 mayimprove matching accuracy under various conditions. In anotherembodiment, the multiple images 30 may be compiled into one fingerprintimage 30 that is a composite of the best portions of the separatemultiple images 30.

At block/step 48, the system 10 may operate in the verification mode. Atthis block/step 48, the system 10 may be directed as to the identity ofthe user 12 that is to be verified, and the system 10 may compare thecaptured fingerprint image 30 with the fingerprint image 30 or images 30previously captured and stored in the database 20 for that particularuser 12. If the captured image 30 matches at least one of the existingimages 30 stored in the database 20 for that particular user 12, thesystem 10 may return a positive-match result to the mobile device 14,thus allowing the user 12 access. If there is no match, the system 10may prompt the user 12 to try to capture the fingerprint again. In otherembodiments, the system 10 may disable the mobile device 14, eitherpermanently or for a set time period, after a preset number of negativeresults and may optionally generate an alert.

At block/step 50, the system 10 may operate in the identification mode.At this block/step 50, the system 10 may not have been informed as towhich previously-enrolled user 12 is attempting to be authenticated.Thus, the system 10 may compare the captured fingerprint image 30 withevery fingerprint image 30 previously captured and stored in thedatabase 20 for all users 12. If the captured image 30 matches at leastone of the existing images 30 stored in the database 20, the system 10returns a positive-match result to the mobile device 14 and allows theparticular user 12 access. If there is no match, the system 10 mayprompt the user 12 to try to capture the fingerprint again. In otherembodiments, the system 10 may disable the mobile device 14, eitherpermanently or for a set time period, after a preset number of negativeresults and may optionally generate an alert.

FIG. 3 shows a mobile device 14 capturing an image 30 of a user's finger22 by a camera 16 located on the rear of the device 14. The camera 16may be located on the front, rear, or side of the device 14. The camera16 may be a CMOS or CCD camera. The camera may have a resolution of atleast two megapixels in another embodiment. The mobile device 14 mayinclude a display screen 26. In another embodiment, the device 14 maydisplay a preview window 24 on the screen 26 during any or all of steps34, 36, 38. In other embodiments of the invention, the preview window 24may be located in the top third of the screen 26 or in the third of thescreen 26 closest to the camera 16. The preview window 24 may display inreal-time what the camera sees, which may be the user's finger 22. Thepreview window 24 may help the user 12 position the finger 22 within thecamera's 16 field of view for capture.

In another embodiment, the preview window 24 may display an overlayguide 28. This overlay guide 28 may be a translucent or transparentobject in the shape of a finger 22. This guide 28 may be positionedwithin the preview window 24 to show the ideal placement for a finger 22for capture of a fingerprint image 30. The guide 28 may be of a shapeand size to represent an average finger 22 placed at an appropriatedistance from the camera 16 to have good depth of field. The previewwindow 24 may simultaneously display the overlay guide 28 and the actualfinger 22 when the finger 22 is placed in front of the camera 16. Inanother embodiment of the invention, the mobile device 14 may mask or“black out” during step 38 everything in the camera 16 field of viewthat is outside the overlay guide 28. In another embodiment, processingof the fingerprint image 30 at step 40 may include cropping the portionof the image 30 that is outside the overlay guide 28.

FIG. 4 shows a flow chart for a method of creating an overlay guide 28from a user's finger image 30 in another embodiment of the invention.The steps of this method may take place during or in parallel with steps38, 40, 42 of the biometric system 10 method. In another embodiment, thesteps of FIG. 4 may take place in enrollment mode. At block/step 72, thecamera 16 may capture an image 30 of the user's finger 22, just as instep 38.

At block/step 74, the captured image 30 may be processed byautomatically cropping the portion of the image 30 that is outside thefinger 22. After isolating the finger 22 in the image 30 and croppingthe background, a customized overlay guide 28 may be created from thecropped finger image 30, and this customized overlay guide 28 may besaved to the mobile device 14 at block/step 76. This custom overlayguide 28 may aid the user 12 by indicating to the user 12 in the previewwindow 24 where to place or locate the finger 22 with respect to thecamera 16 for a good capture. In other embodiments, this custom overlayguide 28 may be used in the same way as the generic, average-fingeroverlay guide 28 described previously with respect to FIG. 3, including,but not limited to, masking or “blacking out” during step 38 everythingin the camera 16 field of view that is outside the custom overlay guide28 and/or cropping the portion of the image 30 that is outside thecustom overlay guide 28 during the processing step 40.

In another embodiment of the invention, the system 10 may return anegative match if a user 12 places a finger 22 at an angle differentfrom that of the custom overlay guide 28; the underlying premise may bethat a user 12 will generally fall in the habit of placing a finger 22before the camera 16 at approximately the same angle each time, so ifthe finger 22 is placed at a different angle, an unauthorized person maybe attempting to gain authentication. In another embodiment, the system10 may analyze the captured fingerprint image 30 and compare it withstored images 30 in the database 20 to determine if the angle of thecurrently-captured fingerprint image 30 is different from the angle ofstored images 30, and the system 10 may return a negative match if theangles are different.

FIG. 5 shows a flow chart for a method of concatenating a user'sfingerprint image 30 with a random image file in a biometric system 10according to another embodiment. The steps of this method may take placeduring or in parallel with step 40 of the biometric system 10 method. Atblock/step 64, the captured fingerprint image 30 of a user's finger 22may be analyzed for significant geographical landmarks of thefingerprint. At block/step 66, these analyzed and extracted significantgeographical landmarks may be converted to a string of data.

At block/step 52, the system 10 may determine in what mode to operate,as in step 44. If the system 10 is in enrollment mode, at block/step 54,the system 10 may assign a unique, randomly-generated image file to theparticular user 12 being enrolled. If the system 10 is in verificationmode, at block/step 56, the system 10 may retrieve from the database 20the random image file previously assigned to the particular user 12being verified. If the system 10 is in identification mode, atblock/step 58, the system 10 may retrieve from the database 20 everyrandom image file in the database 20 previously assigned to every user12. The unique, randomly-generated image file may be of a snowflake inone embodiment; in alternative embodiments, other images may be used.

In another embodiment, the assigned random image file may be stored onthe mobile device 14. Storing the assigned random image file locally onthe mobile device 14 of the user 12 may help decrease the authenticationtime in identification mode if the enrolled profiles are stored in aremote database 20; otherwise, if both the assigned random image filesand enrolled profiles are stored only in a remote database 20, thesystem 10 may need to go through many iterations or loops of theauthentication process described below depending on how many users areenrolled in the database 20.

At block/step 60, the assigned random image file may be analyzed forsignificant geographical landmarks of the random image file. Theanalyzed random image file may be the one assigned to the new user 12 ifin enrollment mode; the one assigned to the user 12 if in verificationmode; or, if in identification mode, every one sequentially oriteratively that is in the database 20. At block/step 62, these analyzedand extracted significant geographical landmarks may be converted to astring of data.

At optional block/step 68, the string of fingerprint significantgeographical landmarks may be concatenated with the string ofsignificant geographical landmarks of the random image file. Atblock/step 70, the captured fingerprint image 30 may be destroyed andnot saved. The data that is transmitted in step 42 may be theconcatenated string from step 68. In such an embodiment, the fingerprintimage is not saved in a database 20 for privacy reasons. Duringverification or identification, the concatenated string may be comparedto the concatenated strings stored in the database 20 for a match.

Numerous embodiments have been described, hereinabove. It will beapparent to those skilled in the art that the above methods andapparatuses may incorporate changes and modifications without departingfrom the general scope of this invention. It is intended to include allsuch modifications and alterations in so far as they come within thescope of the appended claims or the equivalents thereof.

Having thus described the invention, it is now claimed:
 1. A method forbiometric authentication of a user comprising the steps of: a)providing: a mobile device comprising a camera and a screen; and adatabase; b) superimposing on the screen a translucent overlay of afinger over a real-time image as currently seen through the camera; c)capturing with the camera an image of a fingerprint of the user; d)comparing the captured image with an authenticated fingerprint imagethat is stored in the database; and e) returning a positive result ifthe compared images match.
 2. The method of claim 1, wherein, in stepc), the fingerprint image is captured automatically when the mobiledevice detects that a finger is properly positioned before the camera.3. The method of claim 1, wherein a shape and size of the overlay isdetermined by a previously-captured image of the user's finger.
 4. Themethod of claim 3, wherein in step b) the overlay is placed in theone-third portion of the mobile device screen that is nearest to thecamera.
 5. The method of claim 1, wherein step a) further comprisesproviding a server external to the mobile device, wherein the servercomprises the database; the method further comprising steps: f) sendingthe captured image to the server; and g) processing on the server thefingerprint image, which processing includes at least one of: convertingthe fingerprint image to black-and-white; reducing noise in thefingerprint image; reducing blurring of the fingerprint image; enhancingridge contours of the fingerprint image; and compressing the fingerprintimage; wherein step f) occurs after step c) and before step d); andwherein step g) occurs after step f) and before step d).
 6. The methodof claim 1 further comprising steps: h) processing the fingerprintimage, which includes at least one of: converting the fingerprint imageto black-and-white; reducing noise in the fingerprint image; reducingblurring of the fingerprint image; enhancing ridge contours of thefingerprint image; compressing the fingerprint image; and encrypting thefingerprint image; i) calculating a time used for the processing of steph); and j) comparing the calculated time from step i) against a pre-setthreshold; wherein steps h)-j) occur after step c) and before step d);and wherein step e) returns a negative result if the calculated timefrom step i) is less than the pre-set threshold compared against in stepj).
 7. The method of claim 1, wherein the authenticated fingerprintimage that is stored in the database is a compilation of best portionsof multiple authenticated images of the fingerprint of the user.
 8. Themethod of claim 1, wherein the database provided in step a) includesmultiple authenticated images of the fingerprint of the user taken underdifferent ambient conditions.
 9. The method of claim 1, wherein: in stepb), the overlay is oriented such that the finger of the overlay ispositioned horizontally; and in step c), at least one of the top andbottom one-third portions of a field of view of the camera areeither: 1) not imaged, or 2) cropped out after imaging.
 10. A biometricauthentication system comprising: a computer comprising a camera and ascreen; and a database; wherein the system is programmed to: superimposeon the screen a translucent overlay of a finger over a real-time imageseen by the camera; capture an image of a fingerprint of a user with thecamera; compare the captured image with an authenticated fingerprintimage that is stored in the database; and return a positive result ifthe compared images match.
 11. The biometric authentication system ofclaim 10 further comprising an Internet-connected server remote from thecomputer, wherein the server comprises the database.